fix(installer): execute-gate installed tools so a broken binary fails Step 1 (#411) - #451
Merged
Merged
Conversation
… Step 1 (#411) The only post-install "verification" was a log-only, failure-masking interpolation, so a corrupt or wrong-architecture binary (winget shims / partial installs skip checksum verify; brew delivers with no checksum of ours) still printed "System tools" and only died at cluster-create in Step 2. All three OSes had the gap. - New shared execute-gate helpers: assert_tool_runs (common.sh, bash) and Assert-ToolRuns (install-k8s.ps1, PowerShell). Each runs the tool's self-check; on non-zero exit or an exception it removes the located binary and fails loudly with an arch-aware remedy, so the tool step fails instead of the cluster step. - Gate kubectl / k3d / helm after install on Linux (setup-linux.sh), macOS (setup-macos.sh), and Windows (install-k8s.ps1), on both the fresh-install and already-present paths. "System tools" success only prints once all gates pass. - kubectl is gated with `version --client` (NOT --short — removed in kubectl 1.28+, which would false-fail the gate). - New check-drift.sh parity check (_drift_execute_gates) so no installer can silently drop a gate for a tool. Tests: +2 bats (common — working tool passes / broken tool errors + removes the binary), +2 bats (drift parity), +7 Pester (exit-nonzero / exception / binary-removal / arch remedy / static gates). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-gate (#411) The execute-gate (#411) runs `<tool> version` after install, but the setup-linux.bats harness only marked tools "present" via has() — it never provided a RUNNABLE k3d/kubectl/helm. On the toolless CI runner the gate's probe found no binary and failed 4 install_k3d tests; locally it false-passed because a real k3d was on PATH. Add silent (non-recording) runnable stubs to setup() so the gate has something to execute and they shadow any real tool on a dev host. Test-only; no production change.
…te-tools # Conflicts: # scripts/manifest.sha256 # scripts/tests/install-k8s.Tests.ps1
…te-tools # Conflicts: # scripts/manifest.sha256 # scripts/tests/check-drift.bats # scripts/tests/check-drift.sh # scripts/tests/install-k8s.Tests.ps1
shujaatTracebloc
marked this pull request as ready for review
July 28, 2026 15:40
saadqbal
reviewed
Jul 28, 2026
saadqbal
reviewed
Jul 28, 2026
saadqbal
reviewed
Jul 28, 2026
…on; drift extracts calls (review #411) Three review findings on #411 (saadqbal + Bugbot): - Execute-gate deleted the wrong binary on the already-present path. Removal is now OPT-IN via `assert_tool_runs --rm <path>` / PS `-BinPath`, passed ONLY by the fresh-install callers that placed the binary. On the present / brew / winget path we don't pass it, so a broken pkg-managed binary is left in place (deleting a brew symlink just wedges the re-run, and the bad copy may be elsewhere on PATH). bash: --rm on Linux fresh-install, none on present or macOS/brew. ps1: -BinPath tracks the direct-download dest ($null for winget/present). - helm was gated with `version --short`; --short can be dropped like kubectl's was, which would false-fail the gate and (previously) delete a good binary. Use bare `helm version` on all three OSes. - _drift_execute_gates used a whole-file grep that could match comments. It now strips comment lines and matches the actual call (`assert_tool_runs … <t> version` / `Assert-ToolRuns -Name "<t>"`), handling the new --rm form. (no grep -q under pipefail — SIGPIPE would false-fail.) Tests: common.bats split into --rm-removes vs no-rm-leaves; check-drift.bats adds --rm-form + commented-out-gate cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3dc866f. Configure here.
…d it AND it ran (Bugbot #411 r2) Round-1 made removal opt-in but only on the fresh-install path, so a broken installer-placed kubectl/k3d on the ALREADY-PRESENT path (left by a prior run) stayed → `has` true → re-run couldn't self-heal (Bugbot). helm already handled it. Unify all three: callers pass --rm "$TB_TOOLS_DIR/<tool>" (bash) / -BinPath "$TOOL_DIR\<tool>.exe" (ps1) on EVERY path, and the gate removes that path only when the binary that actually ran resolves to it — `command -v … -ef <path>` on bash, `(Get-Command).Source -eq $BinPath` on ps1. So a broken copy we own (fresh or prior-run) self-heals, while a brew/winget/pkg-manager copy elsewhere on PATH is never touched (satisfies the earlier reviewer guard too). Dropped the now-moot $k3dDest/$helmDest null-tracking and helm's -f branch. Tests: common.bats +decoy-copy case; install-k8s.Tests.ps1 removal split into ran-here (removed) vs resolved-elsewhere (left); check-drift already covers --rm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saadqbal
self-requested a review
July 29, 2026 06:20
saadqbal
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Closes #411.
Problem
The only post-install "verification" was a log-only, failure-masking interpolation (
Log "k3d: $(k3d version …)"/… 2>/dev/null || echo present). So a corrupt or wrong-architecture binary — winget shims / partial installs skip the direct path's checksum verify, and brew delivers with no checksum of ours (#429) — still printed✔ System toolsand only died at cluster-create in Step 2. Lukas's audit confirmed it's all three OSes.Fix (both halves + macOS)
assert_tool_runs(common.sh, bash) andAssert-ToolRuns(install-k8s.ps1, PowerShell). Each runs the tool's self-check; on a non-zero exit or an exception it removes the located binary and fails loudly with an arch-aware remedy, so the tool step fails instead of the cluster step.System toolssuccess only prints once all gates pass.version --client, not--short(removed in kubectl 1.28+, which would false-fail the gate).check-drift.shparity check (_drift_execute_gates) so no installer can silently drop a gate.Acceptance criteria
k3d.exefails Step 1 loudly with an actionable message; Step 2 is never reached (Err/errorexits before it).Tests (+11)
common.bats+2 (working tool passes & binary untouched / broken tool errors + removes the binary)check-drift.bats+2 (execute-gate parity)install-k8s.Tests.ps1+7 Pester (exit-nonzero / exception / binary-removal / arch remedy / static gates)Verified locally (CI-identical):
Note
Low Risk
Changes are confined to installer verification and drift tests; behavior is fail-fast with guarded binary removal, with no auth or runtime cluster logic touched.
Overview
Replaces log-only post-install version checks (which could still show ✔ System tools when a binary was corrupt or wrong-arch) with real execute gates that run each tool’s version self-check and fail Step 1 with an arch-aware message before cluster creation.
Adds
assert_tool_runsincommon.shandAssert-ToolRunsininstall-k8s.ps1, wired for kubectl (version --client), k3d, and helm (bareversion, not--short) on Linux, macOS, and Windows for both fresh installs and already-on-PATH tools. Optional--rm/-BinPathremoves only the installer-placed binary when it is the one that failed, so re-runs can self-heal without deleting winget/brew copies elsewhere on PATH.Adds
_drift_execute_gatesincheck-drift.shplus bats/Pester coverage so no platform can drop a gate silently; updatesmanifest.sha256and Linux setup test mocks so gates don’t break install-path assertions.Reviewed by Cursor Bugbot for commit f0f2626. Bugbot is set up for automated code reviews on this repo. Configure here.